Computing and network technologies have transformed many aspects of business and everyday life. The amount and value of data available is increasing rapidly due in part to rapid spread of computers and information technology. Typical large enterprises can have approximately a petabyte of operational data stored in over 1,000 data repositories supporting over 5,000 applications. Data storage volumes grow in excess of 50% annually. Workloads from online transaction processing (OLTP) over large databases are growing at over 60% per year. Repositories for decision support systems, which often contain replicated data, grow two to three times as fast as databases used for OLTP. This growth is expected to continue due to new Web based systems, increased access to existing systems, and the introduction of new sources of data.
It is desirable to be able to manage data collections that are continuously growing. It is also desirable to be able to quickly and efficiently derive pertinent information from large collections of data. Further, it is desirable to discover trends in dynamic data associated with complex processes.